{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "ffb89a23-1310-4d1e-af73-ec7d88131121",
   "metadata": {},
   "source": [
    "# 1、Numpy的深入\n",
    "## 1.1、Numpy的广播\n",
    "### 1.1.1、Numpy的插轴操作\n",
    "插轴的意思是添加一个新的维度，此时不是数据拷贝"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "0e390473-9123-4960-b58d-8a2359fd8029",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[1 2 3] (3,)\n",
      "[[1 2 3]] (1, 3)\n",
      "[[1]\n",
      " [2]\n",
      " [3]] (3, 1)\n",
      "[[1 2 3]]\n",
      "[[1]\n",
      " [2]\n",
      " [3]]\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "X = np.array([1,2,3])\n",
    "print(X,X.shape)\n",
    "X1 = X[None,:]\n",
    "print(X1,X1.shape)\n",
    "X2 = X[:,None]\n",
    "print(X2,X2.shape)\n",
    "X3 = np.expand_dims(X,axis=0)\n",
    "print(X3)\n",
    "X4 = np.expand_dims(X,axis=1)\n",
    "print(X4)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b9ad8ed8-9f67-4863-996f-2437e71aec6f",
   "metadata": {},
   "source": [
    "### 1.1.2、数组之间的运算"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "62263dd7-5f5d-49e5-8acf-53c234f40ef1",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[4 6 5] (3,) (3,)\n",
      "[[2 3 4]\n",
      " [6 7 8]]\n",
      "[[ 6  9  9]\n",
      " [10 13 13]]\n",
      "(2, 3) (3,)\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "a = np.array([1,2,3])\n",
    "b = np.array([3,4,2])\n",
    "c = a+b\n",
    "print(c,a.shape,b.shape)\n",
    "#二维数组如果能够对齐，也是可以运算的\n",
    "a = np.array([[1,2,3],[4,5,6]])\n",
    "b = np.array([[1,1,1],[2,2,2]])\n",
    "d = a+b\n",
    "print(d)\n",
    "# d+c,两行都加信息了\n",
    "print(d+c)\n",
    "print(d.shape,c.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d659491f-c3c0-4fae-bec3-6b4663c19b5e",
   "metadata": {},
   "source": [
    "### 1.1.3、广播的对齐逻辑\n",
    "右对齐，逐个比较，每一位要么相等，要么其中一个是1，(3,1)和(3,4)可以广播，广播的结果是(3,4)\n",
    "如果两个不同维度的数组进行运算，会自动将低维升级为高维，向左添1\n",
    "(2,3)+(3,) ==> (2,3)+(1,3) =>(2,3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "3c634872-d11f-4556-90ec-1cd00acc1a04",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[1. 1. 1. 1.]\n",
      " [1. 1. 1. 1.]]\n",
      "[[2. 3. 4. 5.]\n",
      " [2. 3. 4. 5.]]\n",
      "[[4. 4. 4. 4.]\n",
      " [2. 2. 2. 2.]]\n",
      "****************************************************************************************************\n",
      "[[4 5]\n",
      " [3 4]]\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "a = np.ones((2,4))\n",
    "print(a)\n",
    "b = np.array([1,2,3,4])\n",
    "#a+b\n",
    "print(a+b)\n",
    "c = np.array([[3],[1]])\n",
    "#(2,4) (2,1)==>(2,4)\n",
    "print(a+c)\n",
    "print('*'*100)\n",
    "d = np.array([[1,2,3,4],[3,4,5,6]])\n",
    "e = np.array([[1,2],[2,3]])\n",
    "#(2,4)+(2,2)\n",
    "# print(d+e) #没有对齐，不能进行广播\n",
    "#(2,1) (2,2)\n",
    "print(c+e)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "4f41b276-ae38-4531-953d-95f969a43e02",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[2. 2. 2.]\n",
      " [3. 3. 3.]]\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "a = np.ones((2,3))\n",
    "b = np.array([1,2])\n",
    "#a:(2,3) b:(2,1)==>(1,2)\n",
    "# print(a+b) #两者不能广播\n",
    "#如果希望b可以广播，需要将b转换为(2,1)\n",
    "print(a+b[:,None])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9e364174-c7a1-464c-ac19-77809e89f710",
   "metadata": {},
   "source": [
    "### 1.1.4、广播的练习\n",
    "#### 练习 A｜判断与心算（不写代码）\n",
    "\n",
    "1) 判断是否可广播；若可，给出**结果形状**：  \n",
    "   a. `(5,4)` 和 `(4,)`  \n",
    "   b. `(5,4)` 和 `(5,)`  \n",
    "   c. `(2,3)` 和 `(1,3)`  \n",
    "   d. `(2,3)` 和 `(2,1)`  \n",
    "   e. `(6,)` 和 `(2,3)`  \n",
    "   f. `(3,1)`、`(1,4)` 与 `(3,4)` 三者一起\n",
    "\n",
    "2) 下列哪句能实现**列广播**（`A` 形状 `(m,n)`，`col` 为一维或二维）？  \n",
    "   a. `A + col`（其中 `col.shape == (m,)`）  \n",
    "   b. `A + col[:, None]`  \n",
    "   c. `A + col.reshape(1, m)`  \n",
    "   d. `A + col.reshape(m, 1)`\n",
    "***\n",
    "#### 练习 B｜改错题（让它能跑）\n",
    "```python\n",
    "A = np.ones((2,3))\n",
    "b = np.array([10, 20])      # 目标：按列加到 A 上\n",
    "A + b                        # ← 修改为可运行的写法\n",
    "```\n",
    "***\n",
    "```python\n",
    "A = np.arange(6).reshape(3,2)\n",
    "c = np.array([1,2,3])\n",
    "np.concatenate([A, c], axis=1)  # 想把 c 当行拼到右侧，改法？\n",
    "```\n",
    "***"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "26a2eb82-4cdf-4bb6-b00e-f6191a19bfd8",
   "metadata": {},
   "outputs": [],
   "source": [
    "# a. (5,4) 和 (4,)==>(1,4) 可以 (5,4)\n",
    "# b. a(5,4) 和 b(5,)==>(1,5) 不可以b[:,None] \n",
    "# c. (2,3) 和 (1,3) 可以(2,3)\n",
    "# d. (2,3) 和 (2,1) 可以(2,3)\n",
    "# e. (6,) 和 (2,3) 不能\n",
    "# f. (3,1)、(1,4) 与 (3,4) 三者一起 (3,4) (3,4)可以(3,4)\n",
    "\n",
    "#2) 下列哪句能实现**列广播**（`A` 形状 `(m,n)`，`col` 为一维或二维）？  \n",
    "   #a. `A + col`（其中 `col.shape == (m,)`）  不行\n",
    "   #b. `A + col[:, None]`   可以\n",
    "   #c. `A + col.reshape(1, m)`   不可以\n",
    "   #d. `A + col.reshape(m, 1)` 可以"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "ce1dd145-16f0-40d1-a665-5ab8937e7789",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[11. 11. 11.]\n",
      " [21. 21. 21.]]\n",
      "[[0 1 1]\n",
      " [2 3 2]\n",
      " [4 5 3]]\n"
     ]
    }
   ],
   "source": [
    "A = np.ones((2,3))\n",
    "b = np.array([10, 20])      # 目标：按列加到 A 上\n",
    "print(A + b[:,None]) \n",
    "A = np.arange(6).reshape(3,2) # (3,2)\n",
    "c = np.array([1,2,3]) # (1,3) \n",
    "arr = np.concatenate([A, c[:,None]], axis=1)  # 想把 c 当行拼到右侧，改法？\n",
    "print(arr)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "48dcb970-a697-4a30-98c5-b17adaf65de6",
   "metadata": {},
   "source": [
    "#### 练习 C｜编码题（直接写一行/几行）\n",
    "\n",
    "- 两两和矩阵：给 a(50,)、b(60,)，用广播得到 (50,60) 的两两和。\n",
    "- 两两差矩阵：给 x(100,)，得到 D(i,j)=|x_i-x_j| 的 (100,100)。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "9b037068-8b54-4726-8592-b50b2131c9e2",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[7. 7. 7. ... 7. 7. 7.]\n",
      " [7. 7. 7. ... 7. 7. 7.]\n",
      " [7. 7. 7. ... 7. 7. 7.]\n",
      " ...\n",
      " [7. 7. 7. ... 7. 7. 7.]\n",
      " [7. 7. 7. ... 7. 7. 7.]\n",
      " [7. 7. 7. ... 7. 7. 7.]] (50, 60)\n",
      "[[0. 0. 0. ... 0. 0. 0.]\n",
      " [0. 0. 0. ... 0. 0. 0.]\n",
      " [0. 0. 0. ... 0. 0. 0.]\n",
      " ...\n",
      " [0. 0. 0. ... 0. 0. 0.]\n",
      " [0. 0. 0. ... 0. 0. 0.]\n",
      " [0. 0. 0. ... 0. 0. 0.]]\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "a = np.ones(50)\n",
    "b = np.full(60,6)\n",
    "#两两和矩阵：给 a(50,)、b(60,)，用广播得到 (50,60) 的两两和。(50,1)+(1,60)\n",
    "c = a[:,None]+b[None,:]\n",
    "print(c,c.shape)\n",
    "#两两差矩阵：给 x(100,)，得到 D(i,j)=|x_i-x_j| 的 (100,100)。\n",
    "X = np.ones(100)\n",
    "D = X[:,None]-X[None,:]\n",
    "print(D)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ff45d10a-4ad4-4f54-bc38-57ba3f878111",
   "metadata": {},
   "source": [
    "### 1.1.5、应用举例\n",
    "#### **应用举例1:Z-score讲解**\n",
    "按列 Z-score：X(m,n)，把每列标准化到均值 0、方差 1（不能 for）。主要作用是统一尺度：不同量纲（cm、kg、￥…）放到同一数量级，便于比较，公式如下所示:\n",
    "$z=\\dfrac{x-\\mu}{\\sigma}\\;\\Rightarrow\\; z=0\\;\\Leftrightarrow\\; x=\\mu \\;(\\sigma\\ne 0)$\n",
    "\n",
    "假设有三科成绩\n",
    "- 语文：0–100 分\n",
    "- 数学：0–150 分\n",
    "- 物理：0–200 分\n",
    "创建4位同学的成绩（X 的形状是 (4,3)：4 行学生 × 3 列科目）："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "490a87ff-29a4-4111-99c7-c353ff8ed907",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[ 75. 105. 125.]]\n",
      "[[  5.  15.  35.]\n",
      " [ 15. -15. -25.]\n",
      " [ -5.  30.  25.]\n",
      " [-15. -30. -35.]]\n",
      "[11.18033989 23.71708245 30.41381265]\n",
      "[[ 0.4472136   0.63245553  1.15079291]\n",
      " [ 1.34164079 -0.63245553 -0.82199494]\n",
      " [-0.4472136   1.26491106  0.82199494]\n",
      " [-1.34164079 -1.26491106 -1.15079291]]\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "S = np.array([\n",
    "    [80,120,160],\n",
    "    [90,90,100],\n",
    "    [70,135,150],\n",
    "    [60,75,90]\n",
    "])\n",
    "#(3,)==>(1,3)\n",
    "Mu = S.mean(axis=0,keepdims=True)\n",
    "print(Mu)\n",
    "#完成了量纲的统一\n",
    "print(S-Mu)\n",
    "#计算标准差\n",
    "STD = S.std(axis=0)\n",
    "print(STD)\n",
    "#计算Z-score\n",
    "Z = (S-Mu)/STD\n",
    "print(Z)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "49c04dea-6bea-4f71-870d-d3eab7787db1",
   "metadata": {},
   "source": [
    "#### **应用举例2:归一化的处理**\n",
    "归一化是将数据压缩在0-1的范围内，也是一种统一量纲的方法，和zscore的区别在于，这个能够维持相对比例关系，二zscore会保留相对偏差关系使用的公式如下\n",
    "\n",
    "$x'=\\frac{x - x_{\\min}}{x_{\\max} - x_{\\min}}$\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "2dc5c385-03eb-4a8a-afce-512e891ee852",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[0.66666667 0.75       1.        ]\n",
      " [1.         0.25       0.14285714]\n",
      " [0.33333333 1.         0.85714286]\n",
      " [0.         0.         0.        ]]\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "S = np.array([\n",
    "    [80,120,160],\n",
    "    [90,90,100],\n",
    "    [70,135,150],\n",
    "    [60,75,90]\n",
    "])\n",
    "X_min = S.min(axis=0,keepdims=True)\n",
    "X_max = S.max(axis=0,keepdims=True)\n",
    "X = (S-X_min)/(X_max-X_min)\n",
    "print(X)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "540b39e3-f860-4270-aed2-574978150f83",
   "metadata": {},
   "source": [
    "#### **应用举例3:掩码替换**\n",
    "**“掩码替换”** 它背后的意义，和数据清洗、异常值处理、特征工程都有直接关系。\n",
    "\n",
    "**掩码**：用布尔索引筛出需要修改的元素；\n",
    "\n",
    "**替换**：用对应列的均值替代（通过广播实现列方向对齐）。\n",
    "\n",
    "假设我们认为成绩中 0–20 属于异常低分（可能是缺考、录入错误、测量噪声等），那么可以把 所有落在 [-10,20] 区间 的数值用该科目的均值来替代。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "c161976d-e8bb-41d6-9c4a-4d4a322679a2",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[False False  True]\n",
      " [False  True False]\n",
      " [False  True False]\n",
      " [ True False False]]\n",
      "[[58.5 49.5 87.5]]\n",
      "[[58.5 49.5 87.5]\n",
      " [58.5 49.5 87.5]\n",
      " [58.5 49.5 87.5]\n",
      " [58.5 49.5 87.5]]\n",
      "[[ 80.  120.   87.5]\n",
      " [ 90.   49.5 100. ]\n",
      " [ 70.   49.5 150. ]\n",
      " [ 58.5  75.   90. ]]\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "S = np.array([\n",
    "    [80,120,10],\n",
    "    [90,-9,100],\n",
    "    [70,12,150],\n",
    "    [-6,75,90]\n",
    "],dtype=float)\n",
    "a,b = -10,20\n",
    "#得到每一科的平均值\n",
    "COL_M = S.mean(axis=0,keepdims=True)\n",
    "#得到布尔索引，还是个二维数组\n",
    "mask = (S>=a)&(S<=b)\n",
    "print(mask)\n",
    "print(COL_M)\n",
    "#向下扩展平均值的数量,第一个参数是扩展大小\n",
    "T = COL_M.repeat(S.shape[0],axis=0)\n",
    "print(T)\n",
    "#设置值\n",
    "S[mask] = T[mask]\n",
    "print(S)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "93be79e5-a4fc-44ce-ae20-1635918c3483",
   "metadata": {},
   "source": [
    "#### **应用举例4：图像信息预处理**\n",
    "图像通道标准化（仅 2D/3D 简化版）：img(H,W,3) 与 mean(3,)、std(3,)，完成 (img-mean)/std。这个问题的意义其实和 Z-score 标准化几乎一模一样，只不过对象从“表格中的特征列”变成了“图像的三个颜色通道”。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "138a807d-927d-4dc3-b799-30b6a3b86cd5",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(2, 2, 3)\n",
      "[[[-0.5        -1.         -1.5       ]\n",
      "  [-0.75       -1.33333333 -2.        ]]\n",
      "\n",
      " [[ 0.75        0.33333333  0.5       ]\n",
      "  [ 2.         -1.          3.        ]]]\n",
      "[[[-0.40542855  0.06512605  0.46135076]\n",
      "  [-0.57667609 -0.10994398  0.28705882]]\n",
      "\n",
      " [[ 0.45080914  0.76540616  1.15851852]\n",
      "  [ 1.30704684  0.06512605  2.02997821]]]\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "img = np.array([\n",
    "    [[100,120,130],\n",
    "    [90,110,120]],\n",
    "    [[150,160,170],\n",
    "    [200,120,220]]\n",
    "],dtype=float) #(2,2,3)\n",
    "print(img.shape)\n",
    "#标准值\n",
    "Mean = np.array([120,150,160])\n",
    "Std = np.array([40,30,20])\n",
    "IMG_NOR = (img-Mean)/Std\n",
    "print(IMG_NOR)\n",
    "\n",
    "#深度学习的标准值\n",
    "Mean = np.array([0.485,0.456,0.406])\n",
    "Std = np.array([0.229,0.224,0.225])\n",
    "IMG_N2 = (img/255.0-Mean)/Std\n",
    "print(IMG_N2)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "343196fc-348f-425e-884f-635d51a1333c",
   "metadata": {},
   "source": [
    "#### **应用举例5:加权处理**\n",
    "X(m,n) 与 权重 w(n,)，返回 y(m,) = Σ_j X[i,j]*w[j]（用广播或矩阵乘），加权的意义就是为不同的特征值确定重要性，依然可以使用学生成绩的例子来进行"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "52674591-6307-42c4-9bf5-242769d50d3b",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[32. 48. 32.]\n",
      " [36. 36. 20.]\n",
      " [28. 54. 30.]\n",
      " [24. 30. 18.]]\n",
      "[112.  92. 112.  72.]\n",
      "[[ 80. 120. 160. 112.]\n",
      " [ 90.  90. 100.  92.]\n",
      " [ 70. 135. 150. 112.]\n",
      " [ 60.  75.  90.  72.]]\n",
      "[[112.]\n",
      " [ 92.]\n",
      " [112.]\n",
      " [ 72.]]\n",
      "[112.  92. 112.  72.]\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "S = np.array([\n",
    "    [80,120,160],\n",
    "    [90,90,100],\n",
    "    [70,135,150],\n",
    "    [60,75,90]\n",
    "],dtype=float)\n",
    "w = np.array([0.4,0.4,0.2])\n",
    "print(S*w)\n",
    "T = np.sum(S*w,axis=1)\n",
    "print(T)\n",
    "S1 = np.append(S,T[:,None],axis=1)\n",
    "print(S1)\n",
    "#累加这个过程就是一个典型的矩阵乘法\n",
    "S2 = S@w[:,None]\n",
    "print(S2)\n",
    "#矩阵乘法用得特别的多，所以numpy内部会自动转换\n",
    "S3 = S@w\n",
    "print(S3)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3c364dd2-7686-4138-a463-b331edead49f",
   "metadata": {},
   "source": [
    "## 1.2、文件的读写\n",
    "### 1.2.1、写文件"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "6f656c64-82e7-4fc0-b0ea-26395e22e40e",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[48 64 43]\n",
      " [79 63 55]\n",
      " [88 50 70]\n",
      " [92 74 93]\n",
      " [42 74 54]\n",
      " [74 89 88]\n",
      " [64 55 76]\n",
      " [83 98 56]\n",
      " [99 49 69]\n",
      " [62 42 67]]\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "np.random.seed(100)\n",
    "S = np.random.randint(40,100,size=(10,3))\n",
    "print(S)\n",
    "np.savetxt('score.csv',S,fmt='%d',delimiter=',',header='语文,数学,物理',comments='',encoding='utf-8-sig')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "025ed55f-0d2c-4f7c-a8ff-cd1fbdf92732",
   "metadata": {},
   "source": [
    "### 1.2.2、读文件"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "68406353-7c38-41c4-9d25-8f6d1e89dfe3",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[48 64 43]\n",
      " [79 63 55]\n",
      " [88 50 70]\n",
      " [92 74 93]\n",
      " [42 74 54]\n",
      " [74 89 88]\n",
      " [64 55 76]\n",
      " [83 98 56]\n",
      " [99 49 69]\n",
      " [62 42 67]]\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "S = np.loadtxt('score.csv',delimiter=',',skiprows=1,dtype=int,encoding='utf-8-sig')\n",
    "print(S)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e53a0cde-73bd-49f5-b696-83d3a9a591ca",
   "metadata": {},
   "source": [
    "## 1.3、numpy的矩阵运算\n",
    "### 1.3.1、常用的统计函数"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "1d0c3f76-9e90-4bf7-9f79-cf0dac148bc8",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[48 64 43]\n",
      " [79 63 55]\n",
      " [88 50 70]\n",
      " [92 74 93]\n",
      " [42 74 54]\n",
      " [74 89 88]\n",
      " [64 55 76]\n",
      " [83 98 56]\n",
      " [99 49 69]\n",
      " [62 42 67]]\n",
      "****************************************************************************************************\n",
      "每一科的平均分: [73.1 65.8 67.1] [73.1 65.8 67.1]\n",
      "每一科的标准差: [17.85189066 17.13359273 14.90268432]\n",
      "每一科的最高分: [99 98 93]\n",
      "每一科的最低分: [42 42 43]\n",
      "每一科的极差: [57 56 50]\n",
      "语文的四分位数: [62.5  76.5  86.75]\n",
      "50%的人员的百分比 [76.5 63.5 68. ]\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "S = np.loadtxt('score.csv',delimiter=',',skiprows=1,dtype=int,encoding='utf-8-sig')\n",
    "print(S)\n",
    "print('*'*100)\n",
    "print('每一科的平均分:',np.mean(S,axis=0),S.mean(axis=0))\n",
    "print('每一科的标准差:',np.std(S,axis=0))\n",
    "print('每一科的最高分:',np.max(S,axis=0))\n",
    "print('每一科的最低分:',np.min(S,axis=0))\n",
    "print('每一科的极差:',np.ptp(S,axis=0))\n",
    "print('语文的四分位数:',np.percentile(S[:,0],[25,50,75]))\n",
    "print('50%的人员的百分比',np.percentile(S,50,axis=0))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "30790220-604a-43e8-98d9-863648cae2ce",
   "metadata": {},
   "source": [
    "### 1.3.2、使用IQR剔除异常值"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "9fb4fa15-a061-4ce7-a6dc-a6ae46b46c7b",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "18.0 108.0\n",
      "[ True  True False False False False False False False False False  True\n",
      "  True]\n",
      "[45 48 56 78 65 66 58 78 80]\n",
      "65.0\n",
      "[65. 65. 45. 48. 56. 78. 65. 66. 58. 78. 80. 65. 65.]\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "X = np.array([1,2,45,48,56,78,65,66,58,78,80,110,120])\n",
    "q1 = np.percentile(X,25)\n",
    "q3 = np.percentile(X,75)\n",
    "IQR = q3-q1\n",
    "N = 1\n",
    "lower = q1-IQR*N\n",
    "upper = q3+IQR*N\n",
    "print(lower,upper)\n",
    "#找到异常值\n",
    "mask = (X>=upper)|(X<=lower)\n",
    "print(mask)\n",
    "X1 = X[~mask]\n",
    "print(X1)\n",
    "#np.where可以替换\n",
    "print(np.median(X))\n",
    "M = np.median(X)\n",
    "X2 = np.where(mask,M,X)\n",
    "print(X2)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "48da1430-acb3-4f3b-8c87-83f8f568afca",
   "metadata": {},
   "source": [
    "### 1.3.2、统计函数练习\n",
    "- 基础础练习\n",
    "    - 求每门课的平均分、最高分、最低分。\n",
    "    - 计算每门课的极差、标准差。\n",
    "    - 找出全班语文成绩的中位数与四分位数。\n",
    "- 提高练习\n",
    "    - 判断哪一门课“波动最大”（标准差最大）。\n",
    "    - 对语文成绩排序。\n",
    "    - 求出全班综合平均成绩，并计算每位学生与平均分的差值（Z-score）。\n",
    "    - 对三门课的平均分进行加权求和，算出总评（复习前一节内容）并且添加到最后。\n",
    "    - 使用广播方法让每位学生成绩减去各科平均值（均值中心化)。"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.13.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
